1. Field of the Invention
The present invention generally relates to language learning apparatuses and methods therefor. More particularly, the invention relates to a language learning apparatus and a method therefor for enabling a system to automatically learn languages through interaction with users and an external environment by using speech processing techniques, dialog processing techniques, robot control technologies, sensor technologies, etc. which represent natural language processing techniques.
2. Description of the Related Art
Generally, algorithms for learning languages are used for outputting language knowledge (vocabularies and syntax rules) by using a set of language information and non-language information as an input.
The language information is information which can be inferred during communication, while the non-language information is information which cannot be directly inferred during communication.
An overview of various conventional language learning methods according to the types of given language information and non-language information and the types of language knowledge to be learned is shown in FIG. 7.
According to a first method, by supplying xe2x80x9ca preposition and nouns (text) placed before and after the prepositionxe2x80x9d as the language information and xe2x80x9cthe meaning of the preposition (symbolic representation indicating, for example, the entry number in the dictionary) as the non-language information, xe2x80x9cthe meaning of the preposition dependent on the nouns before and after (symbolic representation)xe2x80x9d is learned as the language knowledge (see C. Harris xe2x80x9cA connectionist approach to the story of xe2x80x98overxe2x80x99xe2x80x9d, Berkeley Linguistic Society, 15, pp. 126-138, 1989).
According to a second method, by supplying xe2x80x9ca preposition, a verb, and a noun (text)xe2x80x9d as the language information and xe2x80x9cthe meaning of a preposition (symbolic representation)xe2x80x9d as the non-language information, xe2x80x9cthe meaning of the preposition dependent on the verb and the noun (symbolic representation)xe2x80x9d is learned as the language knowledge (see P. Munro et al. xe2x80x9cA network for encoding, decoding and translating locative prepositionsxe2x80x9d, Cognitive Science, Vol. 3, pp. 225-240, 1991).
According to a third method, by supplying xe2x80x9ca sentence (text)xe2x80x9d as the language information and xe2x80x9cwhether the reaction made by the system is true or falsexe2x80x9d as the non-language information, xe2x80x9cthe contribution rate of the word for the system""s reactionxe2x80x9d, i.e., xe2x80x9cwhat kind of reaction the system should make in response to a certain word string?xe2x80x9d is learned (see A. L. Gorin et al. xe2x80x9cAdaptive acquisition of languagexe2x80x9d, Computer Speech and Language, Vol.5, pp.101-132, 1991).
According to a fourth method, by supplying xe2x80x9ca sentence (text)xe2x80x9d as the language information and xe2x80x9cthe semantic representation of the symbolic sentencexe2x80x9d as the non-language information, xe2x80x9cthe meaning of the word (symbolic representation)xe2x80x9d is learned as the language knowledge (see J. M. Siskind, xe2x80x9cA computation study of cross-situation techniques for learning word-to-meaning mappingsxe2x80x9d, Cognition, Vol.61, pp.39-91, 1996).
According to a fifth method, by supplying xe2x80x9ca sentence (text) and symbolic vocabulary informationxe2x80x9d as the language information (non-language information is not given), xe2x80x9cthe syntax rulesxe2x80x9d are learned as the language knowledge (see Berwick, xe2x80x9cThe acquisition of syntactic knowledgexe2x80x9d, MIT Press, 1985).
According to a sixth method, by supplying xe2x80x9ca word (text)xe2x80x9d as the language information and xe2x80x9cgraphics (computer graphics)xe2x80x9d as the non-language information, xe2x80x9cthe meaning of the word (graphic pattern) is learned as the language knowledge (see S. Nakagawa et al. An acquisition system of concept and grammar based on combining with visual and auditory information, Transactions of IPSJ, Vol.10, No.4, pp.129-137, 1994).
According to a seventh method, by supplying xe2x80x9ca word (sound)xe2x80x9d as the language information and xe2x80x9cgraphics (computer graphics)xe2x80x9d as the non-language information, xe2x80x9cthe meaning of the word (graphic pattern)xe2x80x9d is learned as the language knowledge (see T. Regier, xe2x80x9cThe Human Semantic Potentialxe2x80x9d, MIT Press, 1997).
According to an eighth method, by supplying xe2x80x9ca sentence (isolated word speech) as the language information and xe2x80x9cwhether the reaction made by the system is true or falsexe2x80x9d as the non-language information, xe2x80x9cthe contribution rate of the word for the system""s reactionxe2x80x9d is learned as the language knowledge (see A. L. Gorin et al., xe2x80x9cAn experiment in spoken language acquisitionxe2x80x9d, IEEE Transactions on speech and audio processing, Vol.2. No.1, pp.224-240, 1994).
In the above-described conventional learning methods, however, when a sentence (text or sound) is given as the language information, and when perceptual information, such as visual information or sensory information, whose meaning is not explicitly given as the non-language information, only the meaning of the word can be learned.
Accordingly, in view of the above background, it is an object of the present invention to provide a language learning apparatus and a method therefor in which, even in response to perceptual information whose meaning is not explicitly given, the syntax structure can be determined and the syntax rules of the input language can be learned based on the determined syntax structure.
In order to achieve the above object, according to one aspect of the present invention, there is provided a language learning apparatus including a recognition portion for receiving language information and for extracting a word string according to the input language information. A semantic analyzing portion receives perceptual information related to the language information and extracts concepts and a concept representation indicating the relevance of the concepts. A relevance analyzing portion verifies the word string extracted by the recognition portion against the concept representation extracted by the semantic analyzing portion and determines a syntax structure of the word string according to the relevance between the word string and the concept representation.
According to another aspect of the present invention, there is provided a language learning apparatus including a speech recognition portion for receiving speech information and for extracting a word string according to the input speech information. A semantic analyzing portion receives perceptual information related to the speech information and extracts concepts and a concept representation indicating the relevance of the concepts. A relevance analyzing portion verifies the word string extracted by the speech recognition portion against the concept representation extracted by the semantic analyzing portion and determines a syntax structure of the word string according to the relevance between the word string and the concept representation.
According to still another aspect of the present invention, there is provided a language learning apparatus including a syntax-structure analyzing unit and a syntax-rule learning unit. The syntax-structure analyzing unit has a recognition portion for receiving language information and for extracting a word string according to the input language information. A semantic analyzing portion receives perceptual information related to the language information and extracts concepts and a concept representation indicating the relevance of the concepts. A relevance analyzing portion verifies the word string extracted by the recognition portion against the concept representation extracted by the semantic analyzing portion and determines a syntax structure of the word string according to the relevance between the word string and the concept representation. The syntax-rule learning unit receives the syntax structure determined by the relevance analyzing portion of the syntax-structure analyzing unit and learns a syntax rule.
According to a further aspect of the present invention, there is provided a language learning apparatus including a syntax-structure analyzing unit and a syntax-rule learning unit. The syntax-structure analyzing unit has a speech recognition portion for receiving speech information and for extracting a word string according to the input speech information. A semantic analyzing portion receives perceptual information related to the speech information and extracts concepts and a concept representation indicating the relevance of the concepts. A relevance analyzing portion verifies the word string extracted by the speech recognition portion against the concept representation extracted by the semantic analyzing portion and determines a syntax structure of the word string according to the relevance between the word string and the concept representation. The syntax-rule learning unit receives the syntax structure determined by the relevance analyzing portion of the syntax-structure analyzing unit and learns a syntax rule.
The aforementioned language learning apparatus may further include a vocabulary-information storage unit for storing vocabulary information for recognizing the language information or the speech information and vocabulary information for recognizing the perceptual information. The recognition portion or the speech recognition portion may recognize the language information or the speech information, respectively, based on the vocabulary information stored in the vocabulary-information storage unit. The semantic analyzing portion may analyze the perceptual information based on the vocabulary information stored in the vocabulary-information storage unit.
The aforementioned language learning apparatus may further include a vocabulary-information storage unit for storing vocabulary information for recognizing the language information or the speech information and vocabulary information for recognizing the perceptual information. The recognition portion or the speech recognition portion may recognize the language information or the speech information, respectively, based on the vocabulary information stored in the vocabulary-information storage unit and may report a vocabulary checked for recognizing the language information or the speech information to the semantic analyzing portion. The semantic analyzing portion may analyze the perceptual information by searching the vocabulary information stored in the vocabulary-information storage unit concerning only the vocabulary reported from the recognition portion or the speech recognition portion.
In the foregoing language learning apparatus, the syntax-rule learning unit may output information of the learned syntax rule to the recognition portion or the speech recognition portion of the syntax-structure analyzing unit. The recognition portion or the speech recognition portion may recognize the language information or the speech information, respectively, based on the vocabulary information stored in the vocabulary-information storage unit and the information of the syntax rule learned by the syntax-rule learning unit.
According to a yet further aspect of the present invention, there is provided a language learning method including: a first extraction step of extracting a word string from language information; a second extraction step of extracting concepts and a concept representation indicating the relevance of the concepts from perceptual information related to the language information; and a determination step of determining a syntax structure of the word string according to the relevance between the extracted word string and the extracted concept representation by verifying the word string against the concept representation.
According to a further aspect of the present invention, there is provided a language learning method including: a first extraction step of extracting a word string from speech information; a second extraction step of extracting concepts and a concept representation indicating the relevance of the concepts from perceptual information related to the speech information; and a determination step of determining a syntax structure of the word string according to the relevance between the extracted word string and the extracted concept representation by verifying the word string against the concept representation.
According to a further aspect of the present invention, there is provided a language learning method including: a first extraction step of extracting a word string from language information; a second extraction step of extracting concepts and a concept representation indicating the relevance of the concepts from perceptual information related to the language information; a determination step of determining a syntax structure of the word string according to the relevance between the extracted word string and the extracted concept representation by verifying the word string against the concept representation; and a learning step of learning a syntax rule based on the determined syntax structure.
According to a further aspect of the present invention, there is provided a language learning method including: a first extraction step of extracting a word string from speech information; a second extraction step of extracting concepts and a concept representation indicating the relevance of the concepts from perceptual information related to the speech information; a determination step of determining a syntax structure of the word string according to the relevance between the extracted word string and the extracted concept representation by verifying the word string against the concept representation; and a learning step of learning a syntax rule based on the determined syntax structure.
In the above-described language learning method, the second extraction step may search only for the concepts of words contained in the word string and may extract the corresponding concept representation.
In the above-described language learning method, the first extraction step may extract the word string based on the learned syntax rule.
In the present invention, the perceptual information may include still image information, moving picture information, touch information, acceleration information, and pressure information.
In the present invention, the syntax rule may include a probability context-free grammar, a modification grammar, and a probability modification grammar.
According to the present invention, language information or speech information is input into the recognition portion or the speech recognition portion, respectively, and a word string corresponding to the input information is extracted.
Meanwhile, perceptual information related to the language information or the speech information, such as moving picture information, still image information, touch information, acceleration information, or pressure information, is input into the semantic analyzing portion, and concepts and a concept representation indicating the relevance of the concepts are extracted from the input perceptual information.
In the relevance analyzing portion, the word string extracted by the recognition portion or the speech recognition portion is verified against the concept representation extracted by the semantic analyzing portion, and the syntax structure of the word string is determined according to the relevance between the word string and the concept representation.
In the syntax-rule learning unit, the syntax rule is learned based on the syntax structure determined by the relevance analyzing portion.
The syntax rule learned by the syntax-rule learning unit may be supplied to the recognition portion or the speech recognition portion, which may then extract the word string based on the learned syntax rule. With this configuration, even if perceptual information whose meaning is not explicitly given is supplied, the syntax structure can be determined. The syntax rule of the input language can also be learned based on the determined syntax structure.
Additionally, in determining the concept representation from the perceptual information, the concepts only corresponding to the words contained in an input word (string) are searched rather than searching all the concepts, and only the corresponding concept representation are extracted, thereby making concept searching more efficient.